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Abstract 

This short note studies a variation of the Compressed Sensing paradigm introduced recently 
by Vaswani et al., i.e. the recovery of sparse signals from a certain number of linear measure- 
ments when the signal support is partially known. The reconstruction method is based on a 
convex minimization program coined innovative Basis Pursuit DeNoise (or iBPDN). Under the 
common ^-fidelity constraint made on the available measurements, this optimization promotes 
the (£i) sparsity of the candidate signal over the complement of this known part. 

In particular, this paper extends the results of Vaswani et al. to the cases of compressible 
signals and noisy measurements. Our proof relies on a small adaption of the results of Candes 
in 2008 for characterizing the stability of the Basis Pursuit DeNoise (BPDN) program. 

We emphasize also an interesting link between our method and the recent work of Davenport 
et al. on the 5-stable embeddings and the cancel-then-recover strategy applied to our problem. 
For both approaches, reconstructions are indeed stabilized when the sensing matrix respects the 
Restricted Isometry Property for the same sparsity order. 

We conclude by sketching an easy numerical method relying on monotone operator splitting 
and proximal methods that iteratively solves iBPDN. 

Keywords: Sparse Signal Recovery, Compressed Sensing, Convex Optimization, Instance Optimality. 

1 Introduction 

The theory of Compressed Sensing (CS) [2J [TU] aims at reconstructing sparse or compressible signals 
from a small number of linear measurements compared to the dimensionality of the signal space. In 
short, the signal reconstruction is possible if the underlying sensing matrix is well behaved, i.e. if it 
respects a Restricted Isometry Property (RIP) saying roughly that any small subset of its columns 
is "close" to an orthogonal basis. The signal recovery is then obtained using non-linear techniques 

"laurent . jacques@uclouvain.be. Research supported by Belgian National Science Foundation (F.R.S.-FNRS). 



1 



based on convex optimization promoting signal sparsity, as the Basis Pursuit DeNoise (BPDN) 
program |10| 15]. What makes CS more than merely an interesting theoretical concept is that some 
classes of randomly generated matrices (e.g. Gaussian, Bernoulli, partial Fourier ensemble, etc) 
satisfy the RIP with overwhelming probability. This happens as soon as their number of rows, 
i.e. the number of CS measurements, is higher than a few multiples of the assumed signal sparsity. 

In this paper we are interested in a variation of the CS paradigm. We assume indeed that the 
support of the signal to recover is partially known, possibly with a certain error. As explained 
in |16} [T7] , this context is indeed well suited to the recovery of (time) sequences of sparse signals 
when their supports evolves slowly over time. In that case, the support of the recovered signal in 
a previous (discretized) time can be used to improve the reconstruction of the signal at the next 
time instance, either by decreasing the required number of measurements for a given quality, or by 
improving the reconstruction quality for a fixed number of measurements. Recovering a signal with 
partially known support is also of interest for certain kind of 1-D signals or images. For instance, 
photographic images, i.e. with positive intensities, have often many non-zero approximation coef- 
ficients in their wavelet decomposition [12]; a prior knowledge that can be favorably used in their 
reconstruction from CS measurements. 

By adapting the proof of [1] , we show in this short note that the recovery algorithm minimizing 
the £i-norm of the signal candidate over the complement of the known support part, i.e. what 
we coin innovative Basis Pursuit DeNoising (iBPDN), has a similar stability behavior than the 
common Basis Pursuit DeNoise program. In particular, this extends the result of |16|I17| to the cases 
of noisy measurements and of compressible signals, i.e. with non-zero but fast decaying coefficients 
in a given sparsity basis. We show also that our method shares somehow the conclusion of the cancel- 
then-recover strategy designed in [9] where Authors propose a recovery algorithm that applies an 
orthogonal projection to separate the measurements into two components, and then recovers the 
known support part of the signal separately from the unknown support component. 

2 Framework and Notations 

Let x = G M n be a sparse or a compressible discrete signal in the sparsity basis VP G M nxn 
of M n , i.e. the vector a G M. n has few non-zero or fast decaying components respectively. For the 
sake of simplicity, we work hereafter with the canonical basis, i.e. \I / = Id, identifying a with x. 
The present work is however valid for any orthonormal e.g. the DCT or the Wavelet basis, by 
integrating \? in the sensing model described in Section [3} 

We now establish some important notations. We write N = {1, • • • , n} the index set of the 
vector components in W 1 . For any vector u G 1", m is the i til component of u with i G Af, us is the 
vector equal to the components of u on the set S C J\f and to elsewhere, while u l , with uppercase 
index I G N to avoid confusion, is the vector obtained by zeroing all but the I largest components 
of u (in amplitude). For non-trivial basis \E', u l would be the best Z-term approximation of u in 
the ^2-norm sense. The complement of any set S C N is denoted by S c = M \ S, and the size 
of S by #5. The £ p norm (for p > 1) of u G W 1 is ||u||| = ]T\ \ui\ p , while its support is written 
suppu = {i G N : U{ ^ 0}. By extension, the £o "norm"Nis defined as ||u||o = # suppu. 



Let us speak now of the prior knowledge that we have on the signal. In addition to the 
1 It is not actually a true norm since for instance it is not positive homogeneous. 



2 



assumption of sparsity or compressibility, we presume that the support of the signal x is partially 
known. In the sequel, we denote the known support part by T C M, while we always refer to 
its size by the letter s = #T. Notice that in our study nothing prevents T to be corrupted by 
some "noise", i.e. a priori T is not fully included to suppx. Moreover, the size of (suppa?) \ T is 
not constrained, what will matter is the values of the components of x on (supprr) \ T, i.e. the 
compressibility of x outside of T. 

3 Sensing Model 

Following the common Compressed Sensing model, our vector x is acquired by a sensing matrix 
$ € M mxn subject to an additional white noise n £ M m , i.e. 

y = <&x + n, 

where y £ IR m is the measurement vector. In this model the noise power is assumed boundecj^jby 
e, ||n|| 2 < e. 

As shown after, even if a part of the signal support is known, the stability of this sensing model, 
i.e. our ability to recover or approximate x from y, is also linked to the Restricted Isometry Property 
(RIP) of the sensing matrix [H [21 [3]. 

Explicitly, the matrix $ £ l mx " satisfies the RIP of order q G N (q < n) and radius < 6 q < 1, 

if 

(1 - S q )\\u\\ 2 2 < ||<H|| < (1 + S q )\\u\\l 
for all q-sparse vectors u S M n , i.e. with ||n||o < q- 

4 Reconstructing on Innovation 

Intuitively, if a part T C N of the signal support is known, a possible (non-linear) reconstruction 
technique of x would simply consist in minimizing the sparsity of a signal candidate u E R n over 
T c , i.e. the ^o- norm of ut c , subject to the common £2 fidelity constraint || — 2/| 1 2 < e as prescribed 
by the noise power bound. As underlined many times in the community, such a procedure would 
result in a combinatorial (NP-hard) problem [13]. Here again an £\ relaxation must be used, with 
possibly additional requirements on the RIP- "conditioning" of $ [151 13] . 

The proposed method is a simple extension of the Modified-CS scheme defined in |16| 117] . 
We integrate indeed the case of corrupted measurements by defining the following optimization 
program, coined innovative Basis Pursuit DeNoising (zBPDN), 

argmin ||ut c ||i s.t. ||y — $u||2 < e. (iBPDN) 

u 

The term "innovative" recalls that this program tries to minimize the sparsity of the signal to be 
reconstructed in the unknown (or innovation) set (supp x) \ T included to T c . 

2 Possibly with high probability. 
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5 iBPDN and £2 — i\ Instance Optimality 



The main result of this note provides the conditions under which the solution of «BPDN is close 
or equal to the initial signal x, i.e. the so-called £2 — £\ instance optimality [6]. It extends in the 
same time the conclusion of |16|. [T7] to the cases of noisy measurements and compressible signals. 

Theorem 1. Under the condition of the sensing model described above, writing #T = s and given 
k G N, let us assume that the matrix $ respects the RIP of order s + 2k with radius <5 s +2fc £ (0, 1), 
and that its radius for the smaller order 2k is 82k 6 (0, 1). Then, if 8\ k + 2<5 s+ 2fc < L iBPDN has 
the £2 — £\ instance optimality meaning that its solution x* respects 

||x-a;*||2 < C Sjfc e + D Sjk e (r; k), 

where r is the residual r = x — xt, and eo(r; k) = /c _1 / 2 ||r — r k \\\ is the compressibility erro^ at 
k-term of r. The two constants C s ^ and D s k, given in the proof, depend on $ only. For instance, 
for small innovation, i.e. when k <C s, if 82k = 0.02 and if8 s +2k = 0.2, C Si k < 7.32 and D s ^ < 3.35. 

Proof. We basically adapt the proof of [1] to signal with partially known support. 

We define the residual r = x — xt, with suppr = (suppx) \ T. Let us write x* = x + h with 
h G W 1 so that the proof amounts to bound \\h\\2- Let To be the support of the k largest coefficients 
of the residual r = x — xt, i.e. To = suppr fc with To n T = 0. 

We define next the sets Tj for j > 1 as the support of the k largest coefficients of h$ c . = h — h$ j 

with Sj = T U Uf=o Ti- By construction, we may observe that we got the partition U/>o ^ = 
(supp x) \ T, with #Tj = k and TjHT = Tj n T f = 0, for j, f > and j ^f. 

Let us write T| = T U To and Tiqi = T U To U T\, with #T| = s + k and #T| i = s + 2k. The 
plan of the proof is to first bound ||/iT| c 01 Ih and then ||/iTj 01 lb- 
Using the triangular inequality, we have ||/iT| c 01 ||2 < Ylj>2 11^3) \U- For j > 1, ||/it.,||i > 

^||^r J+ il|oo by the ordering of the T,-'s, and therefore H^t^iH! ^ ^II^TV+i ||^o ^ fcll^rJIi- This 
leads to 

\HJ- < ^EHHUl = ^ll^lll- (1) 
Since T c = To U T^ and ||n||2 = \\y — <&x\\2 < e, and because x* solves zBPDN, we have 

\\xtA\i > \\ x tc + h T c\\i = \\xt + h To \\i + \\x T c + hTc\\i > \\xt \\i - II^Tolli - ||^t= ||i + ll^rdli, 
and therefore, 

H^rdli < IkHli + II^tJIi + \\hr \\i - \\xt q \\i = 2\\x T ^\\i + \\hr \\i = 2[|r - r To ||i + ||^t ||i- 
Consequently, using ([!]) and the equivalence of the norms £2 and £\ , we get 

H^ 01 || 2 <EllHll2< 2eo(r;fc) + ||HI|2- ( 2 ) 
i>2 

3 It could be called also scaled l\- approximation error. 
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Let us now bound ||%j 01 ||2- Notice that h^ 01 = h — Ylj>2 so that, using Cauchy-Schwarz, 
ll^oilll = ( $/l T|oi' $/lr |oi) 

< ||^T |01 || 2 ||^||2 + E,> 2 K^T |0 i' $ H)l- 

By hypothesis, <3? is RIP of order q and radius 5 q with q £ {2k, s + 2k}. It is proved in [1] 
as a result of the polarization identity, that, for two vectors u and v of disjoint supports and 
of sparsity I and /' respectively, if is RIP of order I + then |(<I>u, <3?f)| < 5[ + [i \\u\\2\\v ||2- In 
addition, since x* is solution of zBPDN and x is a feasible point of its fidelity constraint, ||$/i|| 2 < 
||<3?x* — j/ 1 1 2 + \\y — &x\\2 < 2e. Therefore, combining all these considerations, 

||*/»T| 01 ||2 < 2 V 1 + S s+2k e \\h T]Q1 \\ 2 + \(® h T\ + ®h Tl ,3>h Tj )\ 

i>2 

< 2-^/1 +S s+2 ke ||/lT| 01 ||2 + (^+2fc||^T| ||2 + hkWh^h) X^HII 2 

j>2 

< 2-y/l +5 s +2fce||^T| 01 ||2 + fJ- s ,k II^TioJb^H^TjIb, 

i>2 

With Vs,k = ^s+2k + 6 k- 

Since (1 — 5g+2k)\\hT m III — II^Tioilli) simplifying the last expression and using (pi) lead to 



(1 - 5 s+2 k) ||^T|oill2 < 2-y/l + 5 s+2 ke + n s ,k (2e (r; k) + \\h TQ \\ 2 ), 
or, since \\h To \\2 < ||^T| 01 || 2 , 

||/ir, 01 ||2 < ae + (3e (r;k), 



with a = 2^/1 + ^+2^/(1 - 5 s +2fc - A 4 *,*;) and = 2 /U s , fc / (1 - <W 2 fc - Ms.fc)- 
Finally, using again ([2]), 



/1II2 < II^Tioilb + ||/ir, c m 1 1 2 < ae + (f3 + 2)e (r; k) + ||/tT || 2 < C s ,fc e + D Stk e (r;k), 



01 

with 



1 — ^s+2fc — Ms.fc ' 
and 

= 2 - . 

1 — 0s+2fc — Ms,fc 

The denominator of these two constants makes sense only if 1 — 5 s+ 2k — ^s,k > 0, i.e. if b\ k + 
2b s +2k < 1> which provides the announced reconstruction condition. □ 
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6 Observations 



Some observations may be realized from Theorem [TJ First, in the case where there is no knowledge 
about the signal support, i.e. T = and s = 0, we do find the previous sufficient condition of [I] 
characterizing when BPDN satisfies the £2 — 1\ instance optimality, namely 62k < V2 — 1 as involved 
bySl k + 2S 2k < 1. 

Second, the condition b\ k + 25 s+ 2fc < 1 is satisfied if S s+2 k < V% — 1 since we have always 
$2k < $s+2k- This seems again a simple generalization of the previous result in p], i.e. iBPDN is 
stable if the RIP of is guaranteed over the sparsity order s + 2k with a radius 5 s +2fc < V2 — 1. 
Intuitively, the matrix must be sufficiently "well conditionned" to estimate both the unknown 
values of x on the known set T and the k other significant values of x somewhere outside of T. This 
induces somehow the required s + 2k RIP sparsity order, where s and 2k stand for the degrees of 
freedom of x on T and on T c respectively. 

Third, if the signal x is exactly sparse, there is a k < N — s such that k = #((suppx) \ T) and 
eo(r;k) = 0. Without noise on the measurements, the previous theorem guarantees therefore the 
perfect reconstruction of the signal, 1.6. X — cLS obtained in [16J. 

Finally, the compressibility of the signal x is quantified by the compressibility error eo (r, k) . 
In other words, the compressibility is measured from r = x — xt outside of the known support 
part T of x. This new measure is of course the simple generalization of the previous term eo(k) = 
fc _1 / 2 ||x — = eo(x; k) introduced for instance in pp. 

7 Connection to o^-stable Embeddings and the Cancel-then- Recover 
strategy 

Theorem [l] has an interesting connection with the recent work of Davenport et al. [9] showing 
that several signal processing tasks, i.e. signal detection, classification, estimation and filtering, can 
be realized efficiently on the compressive measurements of a signal without reconstructing it. In 
their work, the Authors study in particular the possibility to subtract from these measurements 
the influence of the known part of the signal support. Let us briefly explain that work before to 
compare our work with this of [9]. 

For this explanation, we use the framework of Section [2] with the simplifying canonical basis 
\P = Id and the pure sensing model y = <&x. We define also the subspace = {u G R™ : 
suppn C T} and the matrix 0, = <3?y £ M mxs , i.e. the restriction of $ to the columns indexed in 
T C M. Two operators can be built from Q, and its Moore- Penrose pseudoinverse & = (0 T f2) -1 Q T , 
i.e. Pq = and Pq± = 1 — fiO^, the orthogonal projectors on the range of O and on the 

nullspace of Q T respectively. 

Writing x = xt + ir c , we can notice that Pq±§x = Pq±$xtc- In short, the influence (or 
interference) of xt on y = Qx may be canceled without reconstructing x. The idea of the cancel- 
then-recover strategy promoted in [9] is therefore to reconstruct actually xt? from y = $x = fe^, 
with $ = Pq±&. This can be done for instance by solving either the Basis Pursuit program 

x = argmin [|it||i s.t. y = $«, 

u 

or an equivalent greedy method as CoSaMP |14[ [9]. Of course, xt = since this part of x does not 
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contribute to the fidelity constraint. It is equivalent to say that the reconstruction runs over the 
space P Id ±M™, where P^j-U = ut c for any u £ W 1 . Therefore, the estimation error between x and 
x can be bounded over T c . 

For this purpose <I> must be characterized in function of <£. This can be done by considering 
a generalization the Restricted Isometry Property: Given 5 £ (0,1) and two spaces U,V C W 1 , a 
matrix <3? realizes a 5-stable embedding of (U,V) if 

(1 - 5) \\u - v\\l < \\$u-$v\\l < (1 + 5) \\u - v\\l, 

for all u £U and v £ V. In particular the RIP of order q and radius 5 q is equivalent to a <5 g -stable 
embedding of (£ 9 ,{0}), with = {« £ P : ||u||o < q} the set of g-sparse signals. The following 
result provides then the desired characterization. 

Lemma 1 (Corollary 4 in [9|). Suppose that $ G R mx " is a 5-stable embedding o/(S 2 fc,ST)- Then 
& is a 5/(1 — 5) -stable embedding of (P ld ±T,2k, {0}). 

In particular, this Lemma implies that if is RIP of order s + 2k with radius <5 s -f 2 fc> it is then a 
<5 s+ 2A;-stable embedding of (E 2 jfc, ^r), and therefore, $ is RIP of order 2k and radius 5 s+ 2k/ (1 — <^s+2fc) 
over the space P Id xM n — The £2 — 1\ instance optimality of the BP program pQ above holds 

if 5' = £ s+2 fc/(l - 5 s+ 2k) < V2 - 1, i.e. if 5 s+2 k < (V% - l)/\/2- In that case, 

||x T c - x T c\\ 2 < Ds'e (x T c,k) = D s >eo(r,k), (3) 



l+fyg-lW _ l+(y / 2-2)5. 



In this paper, we show that iBPDN is optimal when 5\ k + 2<5 s+ 2fc < L This condition is weaker 
than the one proposed in [9j, i.e. <5 s +2fc < (\/2 — l)/y/2, however it is interesting to notice that 
both consider also the RIP of order 2s + k and both are stable for compressible signals. Moreover, 
zBPDN gives guarantees for the estimation of the whole signal and not only for its behavior over 
T c . Of course, if x* is the solution of iBPDN (with e = 0), we get similarly 

\\xtc - Xxc\\2 < ||x-x*|| 2 < D s ^ k e (r,k), 

with D S)k < 2 ; + i^T^ 3+2fc < D 5 , 

We can remark also that, conversely to the current cancel-then- recover strategjQ iBPDN pro- 
vides stability against noisy measurements. An open question is however that $ in [5] has not to 
be really RIP of order s + 2k to valid ([3]). As reported in Lemma [TJ $ simply needs to provide a 
5-stable embedding over (S 2 fe, £t) which is weaker than asking the RIP of order s + 2k. Given k 
and m, that second requirement holds possibly for a smaller radius 5 than the RIP radius 5 s +2k- 



8 Numerical Method 

In this section, we sketch of a simple algorithm for the reader interested in a numerical implemen- 
tation of iBPDN. This one relies on monotone operator splitting and proximal methods 111] . 

4 Robustness of this strategy against an additional noise n could be obtained by bounding the power of PqxTI when 
y — &x + n. 
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At the heart of this procedure is the definition of the proximity operator of any convex function 
ip : M. n — > M, i.e. the unique solution of prox (/ ,(z) = argmin u ^\\u — a? 1 1 § + ^(z). 

Both BPDN and iBPDN are special cases of the general minimization problem 

argmin f(x)+g(x). (P) 

For iBPDN, f(u) = ||iiT c ||i and g(u) = ic(e)( u ) = if n G C(e) and oo otherwise, i.e. the indicator 
function of the closed convex set C(e) = {v € M. n : \\y — §v\\2 < e}- 

Of course / and g are both non-differentiable, however, since (i) their domain is non-empty, 
(ii) they are convex and (iii) lower semi-continuous (lsc), i.e. bruinf,^^, f(u) = /(no) for all no S 
dom/, iBPDN can be solved by the following Douglas- Rachford iterative method 

= (i-a)«( t ) + as 7 op0 (e) (t i ( t )) ) (4) 

where ^4® = 2A — Id for any operator A, at G (0, 2) for all t € N, S y = prox 7 ^ for some 7 > and 
^C(e) = P rox 9 is t ne orthogonal projection onto the tube C(e). From [7], one can show that the 
sequence (u^)t^n converges to some point n* and x* = T , c( e )(u*) is the solution of iBPDN. 

We may compute that 5 7 z = prox 7 ^ z is actually the component- wise soft-thresholding operator 
of z on T c , i.e. (S 1 z)i = sign Zi(\zi\ — 7)+ if i £ T c and z% if i € T, with, for A € M, (A)+ = A if 
A > and else. Efficient ways to compute VcU) are a ^ so given in [TT] . 

9 Conclusion 

This short note has studied the modification of Compressed Sensing introduced in 1 1 6 1. II 7 j . i.e. when 
the signal sparsity assumption is increased by the knowledge of a part of its support. We showed 
theoretically that a simple generalization of the common Basis Pursuit DeNoise program, i.e. the 
innovative BPDN, has similar stability guarantees than BPDN with respect to both signal com- 
pressibility and noisy measurements. Interestingly, the obtained requirements are related to the 
conclusion of [9] when the cancel-then-recover strategy is applied to the context of this paper. 

In the future, we plan to investigate possible numerical applications of this formalism. In 
particular, when iBPDN is integrated to the reconstruction of sequences of sparse or compressible 
signals, we would like to assess the quality of the reconstruction in function of the number of 
measurements when the amount of innovation, i.e. the ratio between the unknown and the known 
signal support parts, can be quantified over time. 
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